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Abstract 



Let C : {0, 1}™ i— » {0, l} m be a code encoding an n-bit string into an m-bit string. Such a code is 
called a (q, c, e) smooth code if there exists a decoding algorithm which while decoding any bit of the 
input, makes at most q probes on the code word and the probability that it looks at any location is at 
most c/m. The error made by the decoding algorithm is at most e. Smooth codes were introduced by 
Katz and Trevisan LK00 in connection with Locally decodable codes. 

For 2-probe smooth codes Kerenedis and de Wolf dWK03 have shown exponential in n lower bound 
on m in case c and e are constants. Their lower bound proof went through quantum arguments and 
interestingly there is no completely classical argument as yet for the same (albeit completely classical !) 
statement. 

We do not match the bounds shown by Kerenedis and de Wolf but however show the following. Let 
C : {0, 1}™ i-> {0, l} m be a (2, c, e) smooth code and if e < then m > 2^~\ We hope that the 
arguments and techniques used in this paper extend (or are helpful in making similar other arguments), 
to match the bounds shown using quantum arguments. More so, hopefully they extend to show bounds 
for codes with greater number of probes where quantum arguments unfortunately do not yield good 
bounds (even for 3-probe codes). 

1 Introduction 

We define smooth codes in a very similar way as defined by Katz and Trevisan |LK00j as follows: 

Definition 1 Let c>l,l/2>e>0 and q be an integer. We call a code C : {0, l} n i— > {0, l} m to be a 
(q, c, e) smooth code if there exists a non-adaptive probabilistic decoding algorithm A such that: 

1. For every x 6 {0, l} n and every i £ [n], we have: 

Pt[A(C{x),i) = Xi]>l-e 



2. For every i S [n] and every j S [m] , we have, 

Pr[y4,(., i) 'reads index'j] < c/m 
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3. In every invocation A reads at most q indices of the code non-adaptively. 

Remark: Here we are considering only non-adaptive codes, however for constant probe codes, bounds 
for non-adaptive codes also imply bounds (up to constants) for adaptive codes [LKOO . 

Katz and Trevisan denned smooth codes in the context of locally decodable codes (ldc) and showed that 
existence of ldcs imply existence of smooth codes. Therefore lower bounds for smooth codes imply lower 
bounds for ldcs. However in our case this is not the case since the error that we are considering is much 
smaller and we are letting the smoothness parameter to be constant. Lower bounds for smooth codes when 
the error is allowed to be constant also imply bounds for corresponding ldcs. 

For smooth codes, Kerenedis and de Wolf idWKOS, using interesting quantum information theoretic 
arguments, showed exponential in n lower bound on m where they let c, e to be constants. This is one of 
the few nice examples of quantum arguments leading to classical results. However till now no completely 
classical argument for the same result is known. Unfortunately the quantum arguments of dWK03 had 
a drawback that they could not be extended to imply similar bounds for smooth codes for higher number 
of probes, for instance these arguments fail to lead interesting bounds even for 3-probe smooth codes. 

We attempt here a completely classical argument for showing exponential lower bound for 2-probe smooth 
codes but we fall short in terms of showing it for constant error. The result we show for smooth codes is 
the following: 

2 n i 

Theorem 1.1 Let C : {0, l} n h-> {0, l} m be a (2, c, e) smooth code and e < Then, m > 2 320^ . 

We hope that although the result here falls short of the desirable, the arguments presented here could 
be extended or similar arguments be made to match, via purely classical arguments, the bounds shown 
by |dWK03] and also more importantly in deriving interesting bounds for codes with higher number of 
probes (in particular for 3-probe codes). 

2 Preliminaries 

In this section we briefly review some of the information theory facts that will be useful for us in our 
proofs in the next section. For a good introduction to information theory, please refer to the fine book 
by Cover and Thomas CT9l|. We let our random variables to be finite valued. Let X, Y be random 
variables. We will let H(X), H(X\Y) represent the entropy of X and the conditional entropy of X given 

Y. We let I(X : Y) = H(X) + H{Y) - H(XY) = H(X) - H(X\Y) represent the mutual information 
between X and Y. We will use the fact I(X : Y) > 0, alternately H{X) + H(Y) > H(XY), alternately 
H{X) > H(X\Y), several times without explicitly mentioning it. We will also use the monotonicity of 
entropy i.e. H(XY) > H(X), alternately H(Y) > I(X : Y) several times without explicitly mentioning it. 
Let X be an m valued random variable, then it follows easily that H{X) < log 2 m (below we always take 
logarithm to the base 2). 

For random variables X\ , . . . , X n , we have the following chain rule of entropy: 

n 

H{X l ,...X n )=Y J H(X i \X l ...X i „ x ) (1) 
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Similarly for random variables X\, . . . , X n , Y, we have the following chain rule of mutual information: 

n 

I(X 1 ...X n :Y) = Y J I{X i : Y\X X . . . X^ x ) (2) 

i=l 

Let X, Y, Z be random variables. Then we have the following important monotonicity relation of mutual 
information: 

I(XY : Z) > I(X : Z) (3) 

All the above mentioned relations also hold for conditional random variables for example, for random 
variables X, Y, Z, I(A : Y\Z) > 0, H(XY\Z) > H{X\Z) and so on. Again we may be using the conditional 
versions of the above relations several times without explicitly mentioning it. 

For correlated random variables X, Y, we have the following Fano's inequality. Let e = Pr[A ^ Y] and let 
\X\ represent the size of the range of X. Then 

H(e) + e\og(\X\-l)>H(X\Y) (4) 
For < p < 1/2, we have the bound H(p) < 

3 Proof of Theorem 11.11 

Let X = X\ . . . X n be a random variable uniformly distributed in {0, l} n (corresponding to the input being 
encoded) and Aj correspond to the i-ih bit of A. This implies that Aj's are distributed independently 
and uniformly in {0, 1}. Let Y = Y\ . . . Y m be a random variable (correlated with A) corresponding to the 
code, i.e Y = C(A). Here Yj,j S [m] corresponds to the j-th bit of the code. 

Let A be as in Definition^ Let < e < 1/2, for i £ [n] let E\ be the graph on [m] consisting of edges 
(j, k) such that, 

Pr[^(C(A),i) = X t \A reads (Yj,Y k )] > 1 - e (5) 
Following interesting fact can be shown using arguments of Katz and Trevisan fLKOO : 

Lemma 3.1 Let C : {0, l} n i— ► {0, l} m be a (2, c, e) smooth code. Let E? e be as described above. Then for 
each i G [n] , Ef e has a matching Mi of size at least ^ . 

Proof: Using the definition of smooth code we have, 



1 - e < Vr[A(C{X),i) = Xi\A{C(X),i) reads E% e ] Pi[A{C(X),i) reads Ef e \ 

+ Pv[A(C(X),i) = Xi\A(C{X),i) reads complement of Ef\ Pr[^(C(A), i) reads complement of Ef\ 
< Pr[^(C(A),i) reads Ef ] + (1 - 2e)(l - Pv[A(C(X),i) reads E? e }) 
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This implies Pv[A(C(X),i) reads Ef\ > 1/2. For an edge e G E? e , let P e = Pv[A(C(X),i) reads e]. This 
implies Y^eeE 2E ^ e — V^- Furthermore since C is a (2, c, e) smooth code, for every j G [to], it implies 
SeG-B 2e |jGe -^e < c/to. Let V be a vertex cover of Ef e . Therefore, 

1/2 < £ ^E E ft<|V|c/m 

This implies that minimum vertex cover of Ef e has size at least m/2c. This now implies that Ef e has a 
matching of size at least to/4c. ■ 

We start with the following claim. 

Claim 3.2 Lei (j, k) G Mj and e' = V8e. Then, I{Xi : YjY k ) > 1 - e'. 
Proof: 

IiXi-.YjYk) = H(X t ) - H(Xi\YiY k ) 

> 1 - H(2e) (from® and ©) 

> 1 — v8e (from the bound < 2^/p) 



We make the following claim which roughly states that the information about various XjS do not quite go 
into the individual bits of Y. For i G [n] let, X% = X\ . . . 

Claim 3.3 

E HXi--Y j \X i ) + I(X i :Y k \X i )<m 
ie[n] (j,k)eMi 

Proof: 

^ ^ J(Xj : Yj\Xi) +I{Xi : Y k \Xi) < ^ ^ J(JQ : Y,-|Xj) (since MjS are matchings) 

ie[n] (J,k)eMi ie[n]je[m] 

= E E J ( x -^i^) 

jG[m] ie[n] 

= ^ I{X : lj) (from ©) 

je[m] 

< to (since Vj G [to] , Yj- is a binary random variable) 

■ 

We now have the following claim which roughly states that for a typical edge (j, k) G Mj there is a 
substantial increase in correlation between Yj and Y k after conditioning on X{. 

Claim 3.4 Let e' < Then, 

^[nW^vAiMiYj : Y k \XiXi) - I(Yj : Y k \Xi)] > 1 - 5c/n 
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Proof: Let (j,k) G Mj. Since Xi and Xj are independent random variables, this implies I(Xi : Xi) = 
and we get: 



I(Xi:YjY k ) < I(X, 

= I(Xi 

= I(Xi 

= I(Xi 

From Claim 1321 we get, 



XiYjY k ) (from ©) 

Xi) + I(Xi : YjYklXi) (from ©) 

YjY k \Xi) 

YjlXi) + I{X, : Y k \Xi) + I{Yj : l^X^) - I(Yj : F,|I t ) (from Q) 



t i (j,k)£Mi 

i (i,fe)eM< 

Claim EQ1 now implies: 

£ 2 J(55 : ni-XiXi) - 7(Y} : Y k \Xi) > (1 - e') ^ |A^| - m 

i (j,k)€Mi i 

> (V |Mi|)(l - e' - f^ttft ) ^ E l^ilX 1 - c / n - 4c / n ) ( from Lemma mi 



Applying Markov's inequality on the above claim we get: 

Claim 3.5 Let < 61,62 < 1- There exists a set GOOD C [n] and sets GOODj C Mi such that: 
1. |GOOD| > (1 - 8 x )n and i G GOOD.E^ k)eMi [I(Yj : - L(Yj : I^Lf*)] > 1 - || 

S. Vi G GOOD,|GOODi| > (1-<5 2 )|M;| andfor(j,k) G GOODLY, : niM)-/^- : F fe Lt;) > 1-^ 

Let 5i = #2 = 1/2. Let e = — . Therefore for i G GOOD and (j, k) G GOODj we have from above, 

: y fc |XiXi) - I{Y j : y fc |X<) > 1 - e (6) 

We fix GOOD to have exactly ^= — 2 elements. For i G GOOD, let a, be the index of i in GOOD. For 
i £ GOOD, let a, be the index of largest i' < i in GOOD. For j G [m],i G [n], let Sj = {I G [m] : 
tf^-IY^X;) < a^e}. Let S? = {j}. 
We show the following main lemma. 

Lemma 3.6 Let i G GOOD, (j, k) G GOOD;. T/ien, 
1. Sf 1 n S^ 1 = 



5c 
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2. Sj^USj -1 ^ 5 j n 4- 
Proof: Part 1: Let I G S 1 *- - D . Using standard information theoretic relations it follows: 

HpWjlYtXi) < H(Y k \YiXi) + HiYjlYX) < 2{a, t - l)e 
Since (J, k) G GOODj and from®, 

H(Y k \Xi) > HiYklXiXi) > I{Y k : Y^X.X) > 1 - e 
Similarly H(Yj\Xi) > 1 — e. Therefore again from®, 

H(Y 3 Y k \Xi) = HiYjlXj + HiY^-IiYj-.YklXi) 
> 2-2e-e = 2-3e 

Now, 

IiYr.YjYklX) = H{YjY k \Xi) - HiYjYklYXi) 

> 2 - 3e - 2(oj - l)e > 2 - 2(oj + l)e > 1 (since a* < i - 2) 

This is a contradiction since y is a binary random variable. 

Part 2: We show Sj -1 US^C Sj and fij -1 US^C S\ follows similarly It is easily seen that C Sj. 
Let Z G S^ -1 . Since (j,k) G GOOD;, from®, 

HiYjlYkXiX) = {{(Y^X) - I(Yj : Y k \XiX) < 1 - (1 - e) = e 

Now, 

= Hftl^iiii) + ^(y.iynXiXi) (from CD) 

< HiYkiYX) + # (y^M) 

< (dj - l)e + e = a;e 



Hence Z G S) and therefore S*" 1 Cfij. ■ 
Our theorem now finally follows. 

Proof: [Theorem II. lj Let i G GOOD. Since e < Claim I3~H holds. Lemma [3.61 implies that for 
(j,k) G GOODj, either |£j.| = 2\S l r l \ or \S\\ = 2\S' l k - l \. Then, 

^logl^l + IGOODil <^log|5j| (7) 
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Let i be the largest i £ GOOD. Now, 

( 2— < 

v 40c 1 8c ~ 

< 

< 
< 

=4> m > 

4 Conclusion 

We have attempted here a classical proof of an already known theorem dWK03 which however has been 
shown using quantum arguments. We hope that the arguments used here are helpful in matching the 
result derived using quantum arguments. The need for a classical proof is also due to the fact that the 
quantum arguments do not help us to derive interesting bounds for codes with higher number of probes, 
in particular even for 3-probe codes. 
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